Desynchronized stable states in diluted neural networks 
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The dynamical properties of a diluted fully-inhibitory network of pulse-coupled neurons are inves- 
tigated. Depending on the coupling strength, two different phases can be observed. At low coupling 
the evolution rapidly converges towards periodic attractors where all neurons fire with the same 
rate. At larger couplings, a new phase emerges, where all neurons are mutually unlocked. The 
irregular behaviour turns out to be "confined" to an exponentially long, stationary and linearly 
stable transient. In this latter phase we also find an exponentially tailed distribution of the inter- 
spike intervals (ISIs). Finally, we show that in the unlocked phase a subset of the neurons can be 
eventually synchronized under the action of an external signal, the remaining part of the neurons 
acting as a background noise. The dynamics of these "background" neurons is quite peculiar, in 
that it reveals a broad ISI distribution with a coefficient of variation that is close to 1. 
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I. INTRODUCTION 

A challenging task in the context of complex neural 
networks is the comprehension of how their dynamical 
features are influenced by their structure. A good strat- 
egy to tackle the problem consists in studying simple 
models, since one can more easily identify the minimal 
and relevant ingredients that are responsible for generic 
properties. In this work we investigate a diluted neural 
network of pulse-coupled neurons. 1 Since inhibition 
plays an important role in the dynamics of neurons in 
vivo 0, 0, we have chosen to examine a network of in- 
hibitory coupled leaky integrate-and-fire neurons. Re- 
cently, it has been shown that dilution can induce long 
chaotic transients in excitatory networks with delay [l3| . 
We aim at understanding the new features emerging in 
the dynamics of a network where two competitive effects 
like dilution and full inhibition coexist. We show that 
the maximum Lyapunov exponent is negative 2 , so that 
the evolution eventually converges onto a periodic attrac- 
tor. This notwithstanding, the dynamics is far from triv- 
ial, because above a given threshold, the transient grows 
exponentially with the system size and is characterized 
by seemingly statonary properties. These features alto- 
gether, plus the stochastic-like nature of the transient, 
suggest a strong analogy with stable chaos (SC), a phe- 



nomenon discovered in the context of coupled map lat- 
tices 0. The possibility of observing SC in the neuro- 
science context is important because this regime is both 
stable to external perturbations and characterized by a 
richness of behaviour depending on the initial conditons: 
both properties are welcome for a system that is expected 
to reliably perform universal computations. The present 
case study can be considered as the first example of SC in 
an autonomous continuous-time system. In fact, it was 
formerly believed that SC can arise only in periodically 
forced systems, as one must simultaneously have a zero 
Lyapunov exponent (in order to ensure a meaningful dy- 
namics different from a dull fixed point) and a strictly 
negative second Lyapunov exponent. The continuity of 
Lyapunov spectra in autonomous spatially extended sys- 
tems prevents the above condition to be fulfilled and, in 
fact, the only previous SC example in a continuous time 
system is a chain of periodically forced Duffing oscillators 
However, these arguments do not apply to globally 
coupled systems and this explains why SC can and has 
been observed in neural networks. 



II. DYNAMICS OF A WEAKLY DISORDERED 
NETWORK 
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1 Here dilution refers to a random pruning of directed links in a 
globally coupled network. 

2 Strictly speaking, it is the second exponent, as there exists an 
exponent equal to zero that corresponds to the motion along 
the trajectory. For the sake of clarity and since we study the 
dynamics emerging from the introduction of a Poincare section, 
we always disregard the zero exponent. 



We consider a system made of N leaky integrate-and- 
fire neurons, which interact by sending each other pulses 
through a directed graph of connections. The non-zero 
elements of the connectivity matrix are Gy = Go jt% , 
where Go is the coupling constant and li is the number 
of incoming links to neuron i . Such a normalization con- 
dition has been used in the literature when dealing with 
randomly connected networks |5lll3|. The membrane po- 
tential vi (t) of the i-th neuron obeys the dynamical rule 
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where we use dimensionless quantities (details about the 
derivation of the model can be found in |l2j). The 
suprathreshold input current, c, and the reversal poten- 
tial, w, are assumed to take the values 2 and 4/7, respec- 
tively. This choice was derived from values used in the 
current literature (see e.g., 0, El). The potential 
follows the above dynamics until it reaches the thresh- 
old value 1, whereupon the neuron emits a spike and the 
potential is reset to the value 0. The integer Mj in the 
coupling term counts the spikes emitted by neuron j at 
the time t^K Since we assume Go > and w > 0, the 
coupling turns out to be fully inhibitory. 

One further important ingredient of the model is dis- 
order that is introduced by randomly pruning a fraction 
r p of directed links. As a matter of fact, the main control 
parameters are r p and the coupling constant Go- 

By introducing a Poincare section (defined as the man- 
ifold where a generic neuron potential reaches the thresh- 
old), it has been shown that the network dynamics can be 
reduced to a discrete map of the potentials Vi (n) between 
successive spikes Accordingly, one can replace the 

continuous time axis with an integer n denoting the nth 
spike. It is convenient to simulate the model by directly 
iterating the resulting map. 

A preliminary analysis has been performed in Ref. Q , 
where it was shown that the evolution converges to a pe- 
riodic state and rigorous upper bounds for the transient 
time have been thereupon derived. Accordingly, the dy- 
namics seems to be quite straightforward and in fact the 
maximum Lyapunov exponent remains negative even in 
the thermodynamic limit [T^ . More precisely, a homo- 
geneous fully coupled network rapidly converges towards 
a periodic phase with all neurons firing with the same 
rate. On the other hand, the transient length may be- 
come exponentially long with the system size, so that in 
large networks the name transient becomes inadequate 
to identify what turns out to be a long-lasting station- 
ary regime. Interestingly enough, upon increasing the 
amount of disorder (i.e., by increasing the fraction r p of 
cut links) one passes from the former to the latter regime. 
The results reported in this section for r p as small as 
0.05, show that for sufficiently large coupling strengths a 
stochastic-like behaviour is self-generated. 

At low coupling strengths (up to Go < 1), the evolu- 
tion is substantially equivalent to that of a fully coupled 
system. After a short transient time, on the order of 
the system size N, all neurons fire with the same pace 
(tisi = const) but are characterized by different absolute 
phases. The main difference between this synchronized 
regime, that we call Locked Phase (LP), and that one oc- 
curring in the fully coupled case is the difference among 
the relative phases. In fact, the equivalence among all 
neurons of a homegeneous network forces the evolution 



to converge towards a highly symmetric state character- 
ized not only by a constant interspike interval (ISI) ii s j, 
but also by a constant time separation between consecu- 
tive spikes. On the other hand, already the introduction 
of a small amount of disorder breaks the symmetry, and 
different spiking sequences are no longer equivalent to 
one another, with the consequence that slightly different 
time series are generated for different initial conditions. 
Although this phase is dynamically trivial, it may have 
some relevance from the point of view of information stor- 
age, since there is an exponentially large (and even more 
than that) number of different attractors that may be 
selected by means of suitable initial inputs. 
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FIG. 1: Firing pattern during the transient in the UP (Go = 
2,7V = 200, r p = 0.05). q denotes the index of the firing 
neuron. 



At higher coupling strengths (Go > 1 ) a different net- 
work dynamics sets in. The ISI is no-longer constant 
and the resulting pattern keeps slowly changing (e.g., see 
Fig. [I). We accordingly call this dynamical regime Un- 
locked Phase (UP) . The phase variables corresponding 
to the different neurons evolve like independent random 
walks, despite the dynamics is characterized by a nega- 
tive maximum Lyapunov exponent. One can understand 
the new scenario by observing that the increased (in- 
hibitory) coupling strength destabilizes (or just makes 
disappear) an increasing number of periodic spiking se- 
quences. Accordingly, we observe a continuous rearrange- 
ment of the ordering, until one of the remaining stable 
sequences is reached. It is certainly desirable to under- 
stand the rules of this pruning of periodic solutions, but 
we first prefer to concentrate our efforts on the statis- 
tical characterization of the transient regime, since the 
irregularity of the evolution can neither be traced back 
to an exponential separation of orbits, nor to the action 
of an external noise source. We start by computing the 
normalized autocorrelation function of the ISIs, t- ls i(n), 
of the single neurons, 
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where (•} denotes the average over the time index n' . 
C; s i (see Fig. |2Jl clearly exhibits an exponential decay, 
while the corresponding ISI distribution (see the inset 
in the same figure) exhibits an exponential tail that is 
typical of Poisson processes and testifies to the absence 
of correlations at long times. 
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FIG. 2: The normalized autocorrelation CW of the t- is i during 
transient for a single neuron vs. the integer delay for TV = 
1000, Go = 2 . The inset shows the corresponding distribution 
P(tisi) that exhibits an exponential decay. 

It is also important to test the stationarity of the UP. 
By computing the coefficient of variation, Cy(n) (the 
standard deviation normalized to the mean value) of the 
ISI in different windows, it can be seen that while Cy(n) 
slowly decreases towards in the LP (indicating a con- 
vergence towards an ordered state), it remains constant 
in the UP 12] although the asymptotic value is quite 
small. For instance, we find that Gy ~ 0.03 for Go = 1.6 
in contrast to values around 1, typically found for ran- 
domly spiking neurons in the cortex This can be 
due to the smallness of the disorder that is considered 
in the network. However, it is also interesting to notice 
that even in the present context for a different setup (i.e., 
under the action of a periodic input) a much wider dis- 
tribution can be found as discussed in the next section. 



III. RESPONSE TO A PERIODIC SIGNAL 

An interesting question is whether the emergence of 
long irregular transients in the presence of multiple pe- 
riodic attractors allows performing neurocomputational 
tasks. As a first test in this direction we study the effect 
of a periodic signal on a subset TV' of the TV neurons. 
The signal is assumed to be an equispaced spike train 
with an excitatory action on the receiving neurons. The 
dynamics of the unforced TV — TV' neurons is still given 
by Eq. (Q, while the remaining TV' neurons are subject 
to the extra excitatory synaptic input, 

M e 

I e = -(y v +w e )G e ^<5(T-4">) ; (2) 



where w c is the reversal potential, G e represents the 
coupling constant and M e counts the external spikes 
received up to the time t . The ISI of the excitatory 
post-synaptic potentials (PSPs) is assumed to be con- 
stant (i.e., Tc™ +1 ' ) — te =T e ). Thus the effective signal 
strength is given by the ratio G e /T e . 

Due to the excitatory nature of the PSP, the firing 
rate of the TV' neurons increases, and this, in turn, gives 
rise to an increased inhibitory activity in the network. 
Thus the activity of the TV — TV' "background" neurons 
that do not partake in the excitatory input should di- 
minish compared to the subset TV' . In Fig. [3] we present 
results for a system of TV = 500 neurons in the UP 
(Go = 1.8). The input signal is switched on during 
the time interval t s [25.9,31.8] ; otherwise the network 
evolves freely according to Eq. QJ. For the PSP input 
trains, we have taken the values w c = —8, T c = 10~ 3 
and G c = 2.2 x 10~ 4 , i.e., G c /T c = 0.22. The input 
is applied to the first 50 neurons (given the equivalence 
on average among all neurons, the choice of the label is 
totally irrelevant); accordingly TV'/TV = 0.1 . By looking 
at the firing pattern shown in Fig.|3Ja), one can clearly 
notice the effect of the excitatory input from the signif- 
icantly increased firing frequency of the driven neurons, 
while, simultaneously, the activity of the undriven back- 
ground neurons diminishes. Not surprisingly, the pattern 
of the TV' neurons is much more regular than the spike 
pattern of the background. This is better seen by look- 
ing at the ISI's, t' isi , of the driven neurons in Fig. E^b). 
Indeed, soon after the input has been switched on the 
t' isi rapidly decrease towards 0.43 with small fluctuations 
< 2 % . Once the signal has been switched off, the system 
rapidly returns to the initial irregular pattern after the 
neurons have been reset. 

When the input signal is on, the forced neurons behave 
almost periodically with a period t' isi 3> T c . The small 
fluctuations are due to the inhibitory action of the back- 
ground neurons. However, the negative Lyapunov expo- 
nent ensures stability with respect to small perturbations 
and thereby guarantees a robust synchronization of the 
subset. The TV' neurons somehow resemble the "active" 
population of cells that respond to a certain stimulus by 
firing with an increased rate. 

On the other hand, the ISIs of the background are more 
irregular than in the freely evolving network as revealed 
by the strongly enhanced fluctuations. In order to per- 
form a more accurate description of their behavior, we 
have switched on the input for longer times, so as to be 
able to determine the probability distribution of the in- 
tervals tjsj in the stationary regime. It turns out that dif- 
ferent background neurons show substantial differences in 
their firing rate, {t- 1S i) 1 . In Fig.^Ja) the distribution of 
(<i S i) is shown, that can be fitted for intermediate values 
by a power-law decay with an exponent ks 3. However, 
the tail shows significant deviations from the power-law, 
which suggests a decomposition of the background neu- 
rons into different classes. In order to get a deeper insigth 
into this phenomenon, we have examined the distribution 
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FIG. 3: (a) Firing pattern for Go = 1.8, N = 500 ; q denotes 
the index of the firing neuron. The excitatory input was app- 
plied to neurons i — 1, ■ ■ ■ 50 for t £ [25.9, 31.8] . To enhance 
visibility only the first 200 neurons are shown, (b) The t- m i of 
the respective firing neuron. With the applied signal the ti B i 
of the receiving neurons lock to t' isi = 0.43 . 




FIG. 4: (a) The stationary distribution of the "background" 
(tisi) for the same parameter values as in Fig. H (circles). The 
solid line is a fit with a power-law with exponent 3. (b) The 
stationary distribution of the Cv of single neurons belonging 
to the "background" for different system sizes (AT'/JV = 0.1). 
(c) The average (Cv) s as a function of the connectivity r con 
for different system sizes (N' /N = 0.1). 



of single neuron Cy's for different system sizes keeping 
N'/N = 0.1 (see Fig. 0[b)). As a matter of fact, this 
characterization of the firing processes reveals a bimodal 
distribution. The peak seen at smaller values is remi- 
niscent of the behaviour of the freely evolving network: 
we can claim that such neurons are almost unaffected by 
the input signal. On the other hand, the neurons with 
Cy ~ 1 exhibit a Poissonian ISI distribution with a much 
more pronounced exponential tail. The different behavior 
of background neurons can be understood by examining 
for each neuron the number of intact incoming links from 
forced neurons, that after a normalization with N' yields 
the local connectivity r con s [0, 1] . A large value of r con 
implies an enhanced inhibition of the respective neuron. 
We collected neurons that share the same value of r con 
and computed the corresponding average Cy that we de- 
note as (CV) S ■ Indeed, the results in Fig.^Jc) show that 
with increasing r con there is a transition-like behavior of 
(Cv) s , as it rapidly increases from values around 0.05 to 
values around 1 . 



IV. CONCLUSIONS AND PERSPECTIVES 

We have studied a diluted inhibitory network of pulse- 
coupled neurons that is dynamically stable in the whole 
parameter range considered. The dynamical stability, 
determined by a negative Lyapunov exponent, ensures 
the existence of asymptotic periodic attractors. By vary- 
ing the coupling strength Go, a desynchronization tran- 
sition is found: for Go ^1-2 the dynamics is governed 
by chaotic-like transients whose duration grows exponen- 
tially with the system size. The transients have good 
stationarity properties and they are characterized by ir- 
regular firing sequences with an exponentially tailed ISI 
distribution. Such a behavior is usually attributed to 
dynamically unstable chaotic networks, e^g., with bal- 
anced excitatory and inhibitory activity |ll| . With the 
help of mean field arguments, the basic mechanism of the 



5 



desynchronization can be identified: the dropout of in- 
put due to a pruned link which leads to irregular changes 
of the neurons position in the firing sequence. Thus we 
expect the effect to emerge in a wider class of models 
(e.g. inhibitory networks with small fractions of excita- 
tory links). Preliminary simulations have further shown 
the effect to persist in the presence of delayed interaction. 

We want to point out that only under the peculiar con- 
ditions of stable chaos (SC) it is possible to reconcile lin- 
ear stability with an unpredictable, chaotic evolution: as 
soon as the system size is sufficiently large, the periodic 
attractor is practically unreachable and the erratic tran- 
sient dynamics is the only accessible information from 
the system. These findings could be of some relevance in 
connection with the recent experimental results of Ma- 
zor and Laurent who pointed out that odor discrim- 
ination is better performed by projection neurons in the 
locust antennal lobe during transient dynamical phases 
preceeding stable states. 

In biological systems, the effect of noise has to be taken 
into account. In the present case this could manifest it- 
self as external background noise or as internal fluctua- 
tions such as synaptic failures. For small enough noise 
amplitudes, the linear stability of the considered system 
ensures robustness of the presented behavior. We have 
shown this by synchronizing a subset of the neurons, 
where the irregular activity of the other neurons can be 
seen as weak background noise. In the case of larger 
noise, one has to consider two cases. In the desynchro- 
nized phase (Go > 1.2) the noise amplifies the already 
present irregularities which renders the convergence to- 
wards a periodic attractor impossible. For what con- 
cerns the locked phase (Go ^ 1-2), we found that finite 
perturbations switch the dynamics between the different 
periodic attractors. However, a detailed investigation of 
noise effects is still in progress. 

The observed mechanisms provided by SC should be 
present in a dynamical model of a system which is ex- 
pected to reliably perform universal computations. In- 
deed, thanks to its asympotic stability, the SC regime 



is found to be naturally stable with respect to external 
perturbations. On the other hand, the long and station- 
ary transient ensures that the system can explore an ex- 
tremely rich variety of configurations. 

This gives rise to the challenging question, whether it 
is possible to address the various periodic states in a re- 
producible way by properly chosen inputs. In this spirit 
a study of the properties of the considered dynamics in- 
cluding external drives (information input) is of high in- 
terest. As a first test, we have demonstrated that it is 
possible to lock by external periodic signals a relatively 
small subset of neurons onto a periodic attractor while 
the rest (background) of the network performs an irreg- 
ular activity. Interestingly, the background decomposes 
into two classes with respect to the Gy of the single neu- 
ron ISI distribution, where one class is characterized by a 
Gy close to unity. All of these remarks indicate that the 
locked as well as the desynchronized phase of our model 
may contain ingredients of direct interest for applications 
to neurosciences. 

Finally, we wish to emphasize that from a conceptual 
point of view, it is remarkable that SC has been found in 
a continuous-time autonomous dynamical system such as 
model After its discovery in space- and time-discrete 
models (coupled map lattices) Q, the only known case 
of a continous-time dynamical model exhibiting SC is a 
chain of periodically forced Duffing oscillators In fact, 
an autonomous dynamical system depending on a contin- 
uous time variable is expected to be marginally stable, 
i.e., A — 0. Such a statement can be made rigorous as 
soon as the interactions which determine the dynamics 
of such a system are assumed to be smooth, differen- 
tiable functions of its variables. This is not the case of 
our model, which is certainly continuous in time but in- 
corporates strong nonlinear, i.e., non-smooth, properties 
due to the threshold mechanism ruling firing events. It 
is this constituent property, typical of neural dynamics, 
which essentially turns the continous time dynamics to a 
map-like behavior, thus rendering possible the emergence 
of SC. 
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